Skip to content

Reflection and comptime goal #311

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

oli-obk
Copy link
Contributor

@oli-obk oli-obk commented Jun 6, 2025

With the renewed general interest in reflection via the facet crate coinciding with my own experiments with reflection, and some discussions with bevy folk at the all hands, I think that this may be a good time to pursue more language reflection capabilities than size_of and align_of.

Rendered

@fu5ha
Copy link

fu5ha commented Jun 7, 2025

Wanted to point out mirror-mirror as another prior art. We made it at Embark Studios for use in our Rust game engine. It's similar to Bevy reflect in many ways, with the most interesting differences being

  • An owned opaque Value type which can be interacted with thru the same reflection apis
  • key paths as a more ergonomic way to query thru layers of a reflected type

@lqd
Copy link
Member

lqd commented Jun 14, 2025

I've added a fix in #312 for the CI failure -- as well as fixed a bug in the template that would still prevent validation without the suggested changes below.


Creating new general purpose crates (like serialization crates, log/tracing crates, game engine state inspection crates) that should work with almost all other data structures is nontrivial today. You either need to locally implement your traits for other crates, or the other crates need to depend on you and implement your traits. This often hinders rollout and will never reach everything.

Reflection offers a way out of this dilemma, as you can write your logic for all types, by processing the type information at runtime (or even preprocess it at compile-time) without requiring trait bounds on your functions or trait impls anywhere.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love the sound of this, but I don't feel like I know exactly what use cases you are trying to enable or aiming to do. Can you give a bit more detail on what kind of things you aim to do? Is the (shiny future) goal to rewrite PartialEq and friends as some kind of fn that is written reflectively instead? Would it be generic over a data structure? Is the expectation that we could 'specialize" it to the type (and get efficiency)? Are you not sure yet, but you know the building block is X?

If you could give a high-level sketch of what these APIs might look like, that'd be great, and bonus points if you can sketch the whole set of connections you eventually imagine.

I don't want this goal to have everything designed up front or anything, I just want to understand the big pieces in your head.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I gave some more details, but

Are you not sure yet, but you know the building block is X?

is a very strong motivation here 😆 I have been wanting some sort of const eval that does type stuff for a long time, and consuming types is much easier than producing new types (which I want to do at some point), so why not start there


## Design axioms

* Prefer procedural const-eval code over associated const based designs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are interesting! I'm having a bit of trouble connecting them to the write-up above, perhaps because I don't quite get what alternatives you are weighing against.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not connect to the write-up above because it is a design axiom for how to write reflection, not why to do reflection. I link to more details below, but TLDR: I do not think uwuflection is the right way forward, as it is a very declarative/functional approach compared to the usual imperative const eval approach we use.

|----------------------|------------------------------------|---------------------------------------------------------------------|
| Lang-team experiment | ![Team][] [lang], [libs] | Needs libstd data structures (lang items) to make the specialization data available |
| Author RFC | | Not at that stage in the next 6 months |
| Lang-team champion | ![Team][] [lang] | TBD |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tmandry would need to assign a champion here

| Design meeting | ![Team][] [lang] | |
| Author call for testing blog post | | Likely will just experiment with bevy or facet, no general call for testing |

## Frequently asked questions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, some of the details I wanted are here, though I had some trouble understanding some of these questions. e.g., "why not go full zig-style comptime" in particular felt like a bit of a nonsequitor.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe because I don't know zig that well


* Prefer procedural const-eval code over associated const based designs
* We picked `const fn` in general evaluation over associated const based designs that are equally expressive but are essentially a DSL
* Ensure privacy is upheld, modulo things like `size_of` exposing whether new private fields have been added
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will privacy be enforced? E.g. for serialization, it is important that private fields can be accessed in serde code. At the moment the privacy boundary is natural as the proc-macro generated code is inside the same module as the struct etc. Would it be possible to give serde access to some private fields, but not other random reflection code? Or is the idea that anything exposed to reflection should be the equivalent of a pub struct with pub fields, and any e.g. deserialisation validation would need to be pushed into a later conversion to a different type? Perhaps we'll need some #[reflect] macro to annotate some fields, though that would open a can of worms of which modifiers from serde and elsewhere to support.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should just punt on this for now and see how far we can get with just public fields. If it turns out you want to be able to create values only via an API that does some sanity checks maybe we'll be able to figure something out, but we'll leave it out of an MVP (I give it a week of the first release of someone sticking data into doc comments on the type that guide their serializer to invoking methods instead of accessing fields 😆 )

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Starting with just "everything must be pub" for the MVP sounds good to me, though it will be useful to keep track of since validation at deserialisation time is a killer feature of serde. Though ... if reflection can inspect (arbitrary) attributes on public fields, then maybe #[reflect(serde::serde(serialize_with = "path"))] could work where reflect would just provide the outer attribute but not parse anything inside (but e.g. serde could then act based on that)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants